Design and development of an RDB version of the Corpus of Spontaneous Japanese

In this paper, we describe the design and development of a new version of the Corpus of Spontaneous Japanese (CSJ), which is a largescale spoken corpus released in 2004. CSJ contains various annotations that are represented in XML format (CSJ–XML). CSJ–XML, however, is very complicated and suffers from some problems. To overcome this problem, we have developed and released, in 2013, a relational database version of CSJ (CSJ–RDB). CSJ-RDB is based on an extension of the segment and link-based annotation scheme, which we adapted to handle multi-channel and multi-modal streams. Because this scheme adopts a stand-off framework, CSJ–RDB can represent three hierarchical structures at the same time: inter-pausal-unit-top, clause-top, and intonational-phrase-top. CSJ–RDB consists of five different types of tables: segment, unaligned-segment, link, relation, and meta-information tables. The database was automatically constructed from annotation files extracted from CSJ–XML by using general-purpose corpus construction tools. CSJ–RDB enables us to easily and efficiently conduct complex searches required for corpus-based studies of spoken language.